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ABSTRACT 

This paper presents an overview of a three-phase 
investigation that featured: (1) an analysis of Office :f Special 
Education and Rehabilitative Services demonstration projects focused 
on employment; (2) consensus development with respect to specific 
employment-related outcomes and activities; and (3) the 
identification of employment-related outcome and activity indicators. 
An emerging systems-level conceptual framework for evaluating program 
activities and outcomes is presented, along with an analytical model 
that illustrates the perceived relationship between an outcome, 
activities associated with producing the outcome, and indicators. 
Phase I results included identification of the outcomes and 
activities taost frequently cited by 42 model demonstration projects 
focused on employment. During Phase II, 22 specific outcomes and 64 
associated activities were identified and socially validated by 106 
model demonstration project directors. Finally, during Phase III, a 
comprehensive list of measures was identified for 17 of the outcomes 
and 51 activities. Results from each phase give substance to the 
analytical model which features specific outcomes and activities at 
multiple levels of impact, supported by both qualitative and 
quantitative indicators. Implications for program restructuring and 
systems change are discussed. (Contains 27 references.) 
(Author/DB) 
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Abstract 

This paper presents an overview of a three-phased investigation that featured (a) an analysis of 
OSERS demonstrations focused on employment, (b) consensus development with respect to 
specific employment-related outcomes and activities, and (c) the identification of employment- 
related outcome and activity indicators. An emerging systems-level conceptual framework (cf. 
Rusch & Phelps, 1987) for evaluating program activities and outcomes is presented also. 
Further, we present an analytical model that illustrates the perceived relationship between an 
outcome, activities associated with producing the outcome, and indicators. Phase I results 
included identification of the outcomes and activities most frequently cited by 42 model 
demonstration projects focused on employment. During Phase II, 22 specific outcomes and 64 
associated activities were identified and socially validated by 106 model demonstration project 
directors. Finally, during Phase III, a comprehensive list of measures was identified for 17 of 
the outcomes and 51 activities. Results from each phase provide substance to the analytical 
model featuring specific outcomes and activities at multiple levels of impact, supported by both 
qualitative and quantitative indicators. Implications for program restructuring and systems 
change are discussed. 
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Promoting Employment for Youths in Transition: 
Outcomes, Activities, and Indicators 

The necessary knowledge, technology, and resources are now available to see the course 
that today's youths with disabilities should take in preparing for the challenges facing them 
tomorrow. If the practices that are being validated by federally sponsored model programs 
throughout the United States could be introduced in every high school, secondary special 
education effectiveness would be drastically improved. 

Clearly, high school experiences are a cornerstone to assuring a youth's success 
throughout life. Indeed, failure to provide an effective high school experience results in 
personal shortcomings, including failure to attain additional education and training to help 
mold a career of personal choice. 

How well are our youths with disabilities making the transition from high school 
student to contributing member of society? How well do high schools prepare students for 
employment? Do students' friendships extend beyond the high school years into early 
adulthood? Do student* with disabilities engage in community activities? 

Ample evidence is available to suggest that our nigh schools fail to achieve desired and 
expected outcomes for all students, regardless of ability (Education Commission of the States, 
1983; National Assessment of Educational Progress, 1990; National Center for Education 
Statistics, 1990; National Commission on Excellence in Education, 1983). For youths with 
disabilities, the unemployment rate exceeds 50% (Wagner, 1989). An even greater percentage 
drop out of high schools each year. Although many of these youths leave school for 
employment, most do not find it. Tragically, tens of thousands of youths with disabilities leave 
high schools without the skills or the support needed to survive independently in their 
communities. Specifically, youths with disabilities who leave school early have less than a 35% 
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chance of finding work (Mithaug, Horiuchi, & Fanning, 1985). Further, youths with disabilities 
live dependently; over 80% live with their parents after high school (Wagner et al., 1991). 

High schools appear primarily to be a training ground only for our nation's most 
promising students— those who will graduate and pursue a college education. Students who do 
not aspire to a postsecondary education, on the other hand, appear to be virtually isolated from 
any unified system that addresses their needs. 

There is, however, reason to be optimistic about the potential for improving secondary 
education in this country. Over the past decade, much research has been sponsored by the 
Office of Special Education Programs, U. S. Department of Education, to address the 
complexities of providing ?n effective secondary special education. Policy-makers, parents, and 
educators from a variety of fields have turned their attention to reform-related issues, including 
the transition from high school to adult life. In particular, much attention has focused on 
programs and practices for promoting employment for youths with disabilities. 

The purpose of this paper is to summarize findings of federally sponsored research 
focused on school-to-work transition with respect to employment-related outcomes and 
activities. Specifically, we present an overview of a three-phased investigation that featured (a) 
an analysis of OSERS demonstrations focused on employment, (b) consensus development with 
respect to specific employment-related outcomes and activities, and (c) identification of 
employment-related outcome and activity indicators. 

Prior to presenting the three phases, we discuss our emerging systems-level conceptual 
framework (cf. Rusch & Phelps, 1987) utilized throughout each phase of the investigation for 
evaluating program activities and outcomes. Further, we present an analytical model that 
illustrates the perceived relationship between an outcome, activities associated with producing 
the outcome, and indicators. Finally, we discuss this research in light of generalizations that 
may be made to all sbidents who are not college bound and who form a large network of 
youths who aspire to graduate and get on with their lives. Most importantly, this paper 
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introduces new knowledge relevant to employment outcomes that has emerged over a 
relatively short time. 

Analytical Framework 
Two analytical models were developed and /or extended during the three-phased 
investigation: (a) a systems-level conceptual framework for evaluating program outcomes and 
activities, and (b) an organizational framework for perceiving relationships between outcomes, 
activities, and indicators. Critical to each phase of the investigation, these analytical models 
provided the framework for analyzing and the structure for presenting and interpreting the 
results. A brief description of the two models follows. 
Systems-Level Conceptual Framework 

Throughout each phase of the investigation, employment outcomes and activities were 
organized according to the systems-level conceptual framework originally conceived by Rusch 
and Phelps (1987) and used subsequently to analyze model demonstration final reports to 
identify project purposes, activities, outcomes, and barriers (e.g., Rusch, Hughes, & Kohler, 
1991). Featuring four levels of possible impact, this framework suggests that programmatic 
outcomes extend beyond the individuals who participate in a particular program. Further, the 
model suggests that programs focused on promoting employment outcomes may have to 
achieve outcomes across multiple levels to produce meaningful, systemic change. 

The four levels include (a) the student and family, most often the primary focus of a 
program or intervention; (b) the program responsible for administering the intervention or 
providing the services; (c) the or ganizations that collaborate w : th the program to provide 
services; and (d) the community, which includes all the generic services, opportunities, and 
barriers that make up the environmental context of a program. The conceptual framework is 
graphically represented in Figure 1. 
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Insert Figure 1 about here 

Or ganizational Framework of Outcomes, Activities, and Indicators 

Efforts to determine program effectiveness have been plagued by a lack of agreed-upon 
outcomes and well-defined activities associated with specific outcomes (Bruininks, Wolman, & 
Thurlow, 1990; DeStefano & Wagner, 1992; Halpern, 1990; Oakes, 1986; Rusch, Kohler, & 
Hughes, 1992). Another problem relates to a need for measures that indicate the level and 
degree of intervention (activities) and achievement (outcomes). Thus, the relationships between 
targeted outcomes and activities, and their indicators, are often ambiguous, lacking clear 
definition and a conceptual framework that illustrates the perceived relationship. 

Our perception of the conceptual arrangement— or organization-between outcomes, 
activities, and indicators is illustrated in Figure 2. This model posits that outcomes are the 
product of action and that particular indicators provide evidence that the outcome has been 
achieved. Further, activities, in the form of specific statements, represent the action associated 
with producing the outcome(s); indicators associated with each activity suggest that the activity 
has taken place. The substance that provides "life" to this model represents the underlying goal 
of the three-phased investigation, that is, the identification of agreed-upon outcomes related to 
employment, activities associated with producing the outcomes, and indicators or evidence that 
the outcome has been achieved and the activities implemented. 

Insert Figure 2 about here 
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Phase I: Analysis of OSERS Demonstrations Focused on Employment 
Faculty and research assistants at the Transition Research Institute at Illinois routinely 
analyze the final reports of model demonstration projects by competition areas to identify 
project demographics and purposes, as well as activities, outcomes, and barriers cited. The 
purpose of Phase I of this investigation was to aggregate the findings from five OSERS 
competitions, four of which focused upon transition from school to work. 

Specific areas of analysis included: (a) examining project variables across competitions 
using the systems-level analytic model; (b) determining the degree to which projects aligned 
themselves with the purposes stated in the OSERS' competition announcements; and (c) 
examining relationships among project purposes, activities, outcomes, and barriers, including 
identifying those variables most frequently cited by projects. [The complete manuscript of this 
study is included in Rusch et al. (1992), the second article in this monograph.] 
Method 

The four employment-focused OSERS competitions examined in the Phase I analysis 
included (a) Research in Education of the Handicapped: Handicapped Children's Model 
Demonstration Projects /Postsecondary Projects (84.023G); (b) Postsecondary Demonstration 
Projects (84.078C); Special Projects and Demonstrations for Providing Vocational Rehabilitation 
Services to Severely Disabled Individuals (84.128A); and (d) Secondary Education and 
Transitional Services for Handicapped Youth: Models for Planning and Implementation of 
Transitional Services (84.158C). 

Data from the analyses of each of the four competition areas (N = 42 projects) were 
aggregated by category (purpose, activities, outcomes, barriers) and organized by conceptual 
level (student/ family, program, organization, community). Subsequently, the variables most 
frequently cited were identified across competitions, by level and category. 
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Results and Discussion 

This early investigation was important for several reasons. Foremost, we applied a 
systems-level conceptual framework to the results reported by federally sponsored transition 
model demonstration projects. Second, we learned that model program directors reported 
many more outcomes related to their demonstrations in 42 communities across the United 
States than originally articulated in OSERS' request for proposals. Finally, this study paved the 
way for a better understanding of the need to relate certain program activities to reported 
outcomes. 

When we compiled the activities reported by the 42 projects we found 88 activities, 
which varied according to their intended impact (e.g., programmatic versus organizational). 
Using the conceptual framework to classify these activities into those levels where they had an 
impact made it easier to identify commonly reported variables within and across competitions. 
Thus, we were able to identify the purposes, activities, outcomes, and barriers cited most 
frequently by the 42 projects (see Table 1). In particular, after classifying activities and 
outcomes, we were able to suggest to the field which activities might be undertaken to increase 
the chances of obtaining selected outcomes. For example, we reported that the outcome most 
frequently cited at the individual level by model program directors was to "Employ 
Individuals" and that these same directors reported "Providing Work Skills Training" as their 
primary activity. Unfortunately, the relationship between these and other outcomes and 
activities could only be implied. 



Insert Table 1 about here 



Another important result obtained from this study related to the shear number of 
activities and outcomes that were reported. For the first time, our field had access to a rich, 
albeit "soft," database consisting of promising practices. We recognized the importance of 
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developing a similar list of outcomes and related activities that would have broader appeal to 
the diverse constituents of transition-related research. Consequently, we decided to launch the 
second phase of our efforts to identify specific outcomes and activities that are considered 
important to achieving employment. Further, we were interested in identifying the specific 
indicators that personnel might use to report these outcomes and activities. 

Phase II: Developing Consensus on Employment Outcomes and Activities 
The purpose of Phase II was to identify specific employment-related outcomes and 
associated activities across the four conceptual levels. Specifically, the study was designed to 
extend the application of the systems-levels framework for evaluating program outcomes and to 
develop consensus on outcomes believed important for student employment after graduation. 
[See Rusch, Enchelmaier, & Kohler (in press), the third article in this monograph, for a complete 
manuscript of this study.] 
Method 

An initial pool of outcomes and activities associated with employment was obtained 
from the list of most frequently cited variables generated in Phase I. This list was incorporated 
into a draft questionnaire and submitted to an expert panel of 12 model demonstration 
transition-to-employment project directors. Based on the panel's feedback, a final questionnaire 
was developed that included 22 employment-related outcomes and 65 associated activities, 
organized by conceptual level (i.e., student/family, program, organization, and community). 
The questionnaire included a 9-point Likert-type scale for rating each outcome and activity (1 = 
Not Important, 9 = Very Important). 

Using a two-round Delphi technique, the questionnaire was sent to 167 transition-to- 
employment project directors who rated each outcome and activity for importance. Two 
months after the first mailing, a second Delphi instrument listing the mean ratings for outcomes 
and activities obtained during Round 1 was sent to the 167 project directors. 
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Results and Discussion 

The consensus-building activities used in this study resulted in a list of 22 outcomes and 
65 related activities. Importantly, we were able to clu^ier these outcomes and activities 
according to their intended focus of impact. Table 2 provides a complete list of the outcomes 
and associated activities organized according to the four conceptual levels (i.e., student/ family 
level, program level, organization level, and community level). As shown, project directors of 
167 federally funded, model transition programs in 42 states believed that the following were 
the five most important outcomes of employment-focused transition programs: (a) utilizing 
individualized education plans (program level); (b) demonstrating improved work 
opportunities (community level); (c) placing students into competitive, integrated employment 
(including supported employment) (individual level); (d) educating students alongside their 
nondisabled peers (individual level); and (e) documenting progress in employment-related skill 
areas (individual level). 

These findings are important because they are the first to suggest an agreed-upon list of 
outcomes that may serve as a blueprint for secondary educators promoting curriculum reform. 
Further, these outcomes suggest that all major participants who work with students with and 
without disabilities may need to acquire different competencies to meet the transition-related 
mandates of the Individuals with Disabilities Education Act Amendments of 1990 (IDEA). 

As we have continued to examine the overall relevance of these findings, we have come 
to recognize important linkages between particular outcomes across the various levels and their 
relevance to the current school restructuring discussion. For instance, not only do personnel 
need to be trained with new outcomes in mind, those responsible for program development 
must address multiple levels of outcomes in order to design more effective high school 
programs. To achieve employment of students (Outcome 1 at the Student and Family Level, 
Table 2), high schools need to plan individualized programs that: include job-skills training, 
assess student growth and change., and ensure provision of related services from year to year 
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(Outcomes 7, 8, 5, and 6, respectively, at the Program Level). Additionally, high schools must 
be cost-effective (Outcome 14 at the Organization Level) and impact services an<* opportunities 
within their communities (Outcomes 21 and 22 at the Community Level). 

Insert Table 2 about here 



This study was important also because it increased our understanding of outcomes. As 
we pursued this research program, we were continually reminded of our failure to consider the 
"bigger picture*' and found that a certain degree of ambiguity skill exists related to outcomes 
and their activities. Consequently, we decided to pursue a more complete model of outcomes 
and activities and their indicators. Our expectation was that if we were able to engage the field 
in "defining" these outcomes and indicators, we would see a more complete transition systems- 
change model appear, a model emphazing the individual and family the high school program 
and cooperating agencies, and the relationships that exist to promote diversity in our 
communities. 

Phase III: Identification of Employment-Related 
Outcome and Activity Indicators 

The purpose of Phase III was to identify potential measures for evaluating the multiple 
outcomes and activities thought important for promoting employment of youths with 
disabilities. Specifically, this study sought to extend the results of phases I and II by identifying 
indicators for 17 of the employment outcomes and 51 associated activities identified and socially 
validated during Phase II. These 17 outcomes received a mean rating of 7.00 or higher during 
the Delphi procedure. 

Thus, the intent of Phase III was to provide the substance necessary to complete Figure 
2, the organizational framework of outcomes, activities, and indicators. That is, the focus was 
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on extending the systems-level model that featured outcomes and activities across the four 
conceptual levels, identifying measures to show that an outcome at a particular level had been 
achieved, identifying activities associated with producing the outcomes, and finally, identifying 
measures to indicate that an activity had taken place. [For a complete description of this phase 
01 the investigation, see Kohler and Rusch (1993) the fourth article in this monograph.] 
Method 

Results from Phase II were mailed the 167 project directors who served as the 
participant pool for the Delphi procedure. Fifty-three of them noted that they were interested in 
identifying measures for the outcomes and activities. They subsequently identified the five 
outcomes they were most interested in focusing on. Based on this information, participants 
were assigned to identify measures for specific outcomes and the activities associated with 
them. To facilitate data collection, the Delphi instrument wa& modified to include space for 
listing ways to measure each outcome and each activity. The five outcomes with a mean rating 
of less that 7.00 were not included in the instrument. 

Responses were received for each of the 17 outcomes and all related activities. As 
responses were received, content analysis was conducted on the suggested measures. 
Redundant measures were removed and a comprehensive list of indicators for each outcome 
and activity was generated. This draft list was subsequently mailed to the initial pool of 167 
project directors for feedback. 
Results and Discussion 

Both qualitative and quantitative indicators were submitted by the 30 project directors 
who participated in Phase III. For instance, qualitative measures suggested to reflect that 
students had been placed in competitive integrated employment (Outcome 1) included job 
types, description of benefits, and job patterns. Quantitative measures for the same outcome 
included hours worked, hourly wages, number of students placed, and student demographics. 
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Table 3 presents one outcome from each conceptual level, activities associated with that 
outcome, and suggested measures for each (see Appendix for a complete listing). 



Insert Table 3 about here 



This study has important implications for advancing theory and for extending a better 
understandmg of practices related to employment. Our intent was to extend both the systems- 
change and the analytical models we have been studying. The systems-change model we 
propose recognizes the importance of multiple "players" at several levels to promote the 
transition of youths into the workplace. Transition from school to work requires that students 
receive relevant experiences in work settings that invite participation by all youths, regardless 
of their diversity from the mainstream. The analytical model operationalizes our "theory" into 
practice by providing multiple audiences with a "blueprint" that describes their roles in building 
the foundation for substantive change in the lives of young people who want to meet their own 
and others' expectations for being involved in their communities. 

Related to practices, this study enjoined the participation of transition experts in an 
activity that invited them to acknowledge their understanding of the theoretical and practical 
importance of our research program. Their participation resulted in one of the most complete 
lists of transition strategies available. We were interested in completing our self-imposed 
obligation to identify indicators related to selected outcomes and activities. We gained much 
more: In our zeal to identify indicators, a list of strategies associated with each activity resulted. 
For example, the first activity listed under Outcome 6 (Utilize transition planning) suggests that 
we "develop strong cooperative linkages with vocational rehabilitation services to develop the 
lEPs." On face value, "develop strong cooperative linkages" sounds like more academic 
rhetoric. However, when 10 indicators were identified by the participants in this study, we 
learned that "cooperative linkages" relates to "Number of Agreements Signed, Number of 
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Consultations Documented, Participation of Personnel/' and so on (see Table 3). Thus, as a 
result of this third study, we now possess a number of strategies (indicators) that can be utilized 
to implement the activity and measure whether it has taken place. 

General Discussion 

This investigation featured three phases: (a) an analysis of OSERS demonstrations 
focused on employment, (b) consensus development with respect to specific employment- 
related outcomes and activities, and (c) identification of employment-related outcome and 
activity indicators. Most importantly, we identified actual program outcomes and activities 
and used them as a basis for developing a conceptual framework of outcomes, activities, and 
indicators for promoting employment of youths with disabilities. This conceptual framework 
received substance from a national group of stakeholders presenting multiple perspectives. 
Thus, the results offer a realistic model that has the potential to serve as a blueprint for systems 
change. 

To date, many recommendations with respect to "promising" practices related to 
promoting employment have been based on implications of research rather than research findings 
(Kohler, 1993). Thus, recommendations to districts scurrying to implement the transition- 
related mandates of IDEA are characterized by a great deal of rhetoric. We too have been 
"guilty" of advising service developers to "implement transition planning," "cooperate with 
community agencies," " place students into competitive integrated employment," and "improve 
access to community services" without offering specific strategies for doing so. Of course, 
implementation strategies will vary to some degree, dependent upon the resources and the 
context of a community. But, as a result of this investigation, we can offer a model of outcomes 
and activities related to promoting employment that includes a comprehensive array of 
strategies, or indicators. Two significant implications are inherent in this model. 

First, the model can be used by those seeking to establish education and community 
programs that focus on employment of youths with disabilities. Again, how program builders 
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specifically implement part or all of the model depends on the local context, but in designing 
new or restructuring old programs, a number of strategies are available for them to consider. In 
particular, the model will be useful in three phases of program development: planning, 
implementation, and evaluation. For planning purposes, the employment outcome model can 
serve as a blueprint for identifying target outcomes across multiple levels. During 
implementation, the activities and indicators components serve as the action plan for specific 
program tasks. Finally, data collection associated with selected indicators can be used to 
provide formative evaluation of program implementation and to provide summary information 
on whether or not targeted outcomes and activities have been achieved. Based upon evaluation 
findings, program improvement, expansion, and/or replication can be considered. Thus, such a 
model can empower service developers to act, evaluate, and react with respect to promoting 
employment for youths with disabilities. 

Second, the employment outcome model can serve as a research tool for seeking 
substantive evidence in support of specific activities and outcomes related to employment. A 
number of studies have been conducted to identify the status of youths with disabilities after 
they have exited high school (e.g., de Bettencourt, Zigmond, & Thornton, 1989; Fardig, 
Algozzine, Schwartz, Hensel, & Westling, 1985; Haring, Lovett, & Smith, 1990; Hasazi, Gordon, 
& Roe, 1985; Mithaug, et al, 1985; Roessler, Brolin, & Johnson, 1990; Wagner, 1989). However, 
fewer studies have actually identified evidence that supports a direct linkage between particular 
practices and targeted outcomes (Kohler, 1993). 

Recently, various researchers have proposed conceptual models that articulate patterns 
of influence among independent and dependent variables to guide investigations of 
relationships between student characteristics and experiences, program characteristics and 
contexts, and student and program outcomes (cf. Halpern, 1993; Harnisch, Wermuth, & Zheng, 
1992; Wagner, 1991). Others have articulated the need for a conceptual model for developing 
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that they restructure the delivery of educational experiences and related services in line with 
these outcomes. Concurrently, we must gather specific data about what we are doing and how 
we are succeeding. By implementing the strategies identified in this investigation, and 
evaluating subsequent outcomes, we can hope to diminish the prospects of an uncertain future 
that face so many individuals. 
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This research was sponsored in part by the Office of Special Education and Rehabilitative 
Services (OSERS), U. S. Department of Education, under a cooperative agreement (H158- 
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reflect those of OSERS. 
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Table 1 



Summary Table of Mo st Frequently Cited Purposes, Activities, Outcomes, and Barriers of 42 
OSERS Funded Employment-Focused Projects 



Conceptual 
Level 


Purposes 


Activities 


Outcomes 


Barriers 


Student and/or 
Family 


Improve 
Vocational 
Training 
(30) 


Provide Work 
Skills 
Training 
(24) 


Employ 
Individuals 
(24) 


Parent and /or 
Family 
Resistance 
(8) 


' Program 


Establish 
Community-Based 
Model Programs 
(9) 


Implement 
Programs or 

Materials 
and Evaluate 
Effectiveness 
(42) 


Establish 
Employment 
Training Programs 
or Services 
(17) 


Personnel 
Issues 
(10) 


Organization 


Develop 
Cooperative 
Delivery Systems 
(19) 

Disseminate 
Information 
04) 


Enhance 
Interagency 
Collaboration 
or Cooperation 
(21) 

Disseminate 
Information 
(29) 


Establish 
Cooperative 
Delivery Systems 
(18) 

Disseminate 
Information 
(31) 


Lack of 
Collaboration 
or Cooperation 
(10) 


Community 


None Cited 
(40) 

Enhance Public 
Awareness or 
Policy 
(2) 


None Cited 
(28) 

Conduct Public 

Relations 
Activities or 
Training 
(10) 


None Cited 
(42) 


Transportation 
(12) 



Note . From "An Analysis of OSERS-Sponsored Secondary Special Education and Transitional 
Services Research" by F. R. Rusch, P. D. Kohler, and C. Hughes, 1992, Career Development for 
Exceptional Individuals, 15, p. 137. Copyright 1992 by the Division on Career Development and 
Transition of the Council for Exceptional Children. Reprinted by permission. 



22 



Promoting Employment 
23 



Table 2 

Mean Rating Values for 22 Employment Outcomes and Associated Activities 

Outcome Activity 



Mean Haan 



Nuaber Number Description Round 1 Round 2 



Student and Family Level 

1 Model trans ition-to-employment projects must place 8.28 8.40 

students into competitive, integrated employment 
(including supported employment). 

1.1 Provide job placement services. 7.91 7.85 

1.2 Work with adult service agencies to ensure job 7.66 7.72 
p lacement . 

1.3 Provide job exploration and job- training opportunities 8.25 8.50 
as part of the school curriculum to prepare students 

for competitive employment. 

1.4 Provide job support services. 8.09 8.27 

1.5 Provide the technical assistance to adult service 6.88 6.55 
agencies to provide job placement and job support 

services . 



ERIC 



Model transition-to-employment projects should 
demonstrate functional skill development of students. 

2.1 Use instruments and procedures that identify 
individual functional skills and consumer 
preferences and life goals. 

2.2 Develop individualized objectives for students that 
reflect functional skill development in the domains 
of vocational skills, independent living, and 
community integration. 

Model transition-to-enrployment projects should ensure 
that students experience education or training with 
nondisabled peers. 

3.1 Conduct training activities for youths without 
disabilities as well as for those with disabilities. 

3.2 Utilize integrated competitive and supported 
employment placements. 

3.3 Utilize nonpaid volunteer placements in compliance 
with Department of Labor (DOL) standards. 

3.4 Utilize community -based education and training sites. 



8.04 8.09 



7.51 7.66 



8.09 8.32 



8.29 8.39 



6.47 6.28 



8.59 8.68 



6.51 6.31 



8.26 8.47 
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Outcome Activity pfean Mean 

lumber number Description Round 1 Round 2 



Program Level 

4 Model transition-to-employment projects should upgrade 6.65 6.08 
the skills of professionals and paraprofessionals to 

licensing standards at their local equivalent. 

4.1 Train vocational counselors. 6.76 6.41 

4.2 Train job coaches. 7.64 7.48 

4.3 Provide inservice training for secondary and 8.04 8.08 
postsecondary teachers including job developers, 

resource teachers, and vocational educators. 

4.4 Train instructors on community, functional 7.74 7.84 
skill-based curriculum and instructional methods. 

5 Model transition-to-employment projects should 7.12 7.39 
publish reports of their students 1 documented 

progress. 

5.1 Develop a final report for students and their 6.87 6.83 
families. 

5.2 Utilize Individualized Education Plans (IEPs) as 5.71 5.52 
the basis for publishing reports. 

5.3 Conduct a longitudinal study of graduates and report 7.50 7.61 
these data. 

6 Model transition-to-employment projects should be 8.25 8.41 
continued beyond the federal funding period. 

6.1 Identify alternative funding from other agencies to 7.92 8.08 
continue the project. 

6.2 Develop funding from within local special education 7.99 7.93 
program budget for transition-to-employment projects. 

7 Model transition projects should utilize individualized 8.45 8.77 
transition planning for students with disabilities. 

7.1 Develop strong cooperative linkages with vocational 7.85 7.98 
rehabilitation services to develop IEPs. 

7.2 Develop strong cooperative linkages with vocational 7.55 7.74 
education services to develop IEPs. 
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Outcome Activity Hean Mean 

Nuaber Nuaber Description Round 1 Round 2 

8 * Model transition-to-employment projects should 7.76 8.01 

provide job-skill training. 

8.1 Conduct job-skill analysis. 7.53 7.90 

8.2 Develop a curriculum to facilitate training. 7.12 7.36 

8.3 Determine labor-market needs. 7.45 7.41 

8.4 Identify job skills that employers require of their 8.03 8.32 
employees. 

9 Model transition-to-employment projects should 8.10 8.34 
document student progress in employment-related 

skills (e.g., social skills). 

9.1 Undertake social skills assessment of students. 7.32 7.54 

9.2 Provide on-site community-based training in 7.85 8.02 
employment- related social skills. 

9.3 Assess student progress in job-related skills. 8.06 8.24 



10 Model transition-to-employment projects should 

achieve replication at least at the level of full 
utilization of a project feature, component, or 
product (such as a training manual). 

10.1 Negotiate directly with an organization or agency 
to replicate the model project. 

10.2 Develop and disseminate replication manual(s). 
10.3 



11 Model transition-to-employment projects should 

establish employment support services. 

11.1 Train job coaGhes. 

11.2 Provide co-worker training. 



9 

EMC BEST COPY AVAILABLE 



7.33 7.40 



6.54 6.42 



7.05 7.01 



Disseminate information and products to other 7.61 7.73 

agencies . 



7.77 7.65 

7.39 7.56 
7.09 7.41 



11.3 Utilize vocational rehabilitation counselors as 7.29 7.75 

appropriate. 
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Outo 
Number 



ActiTity 
Number 



Description 



Moan Mean 

Round 1 Round 2 



12 



11.4 Educate employers about benefits of hiring 
people with disabilities. 

11.5 Train and utilize job developers 

Model transition- to -employment projects should 
develop curricula. 

12.1 Conduct an analysis of job coach roles. 

12.2 Employ personnel with curricula-writing skills. 

12.3 Review existing curricula to determine if new 
curricula are needed. 



7.75 

7.60 
6.43 

6.14 
5.36 
6.76 



8.08 

7.66 
6.38 

6.26 
5.28 
6.76 



12.4 Conduct needs assessment to determine curricular 

needs . 



6.60 



6.51 



Organization Level 
13 



14 



13.1 



13.2 



Model transition-to-employment projects should 
develop materials to facilitate replication 
(e.g., replication guides, training manuals, 
assessment instruments). 

Allocate a section of the project budget to 
publication and production costs. 

Identify production priorities initially and monitor 
throughout the life of the project. 

Model transition-to-employment projects should 
demonstrate cost effectiveness. 



7.29 



7.26 



6.90 



7.45 



7.12 



7.20 



6.76 



7.72 



15 



14.1 Record all real costs of project activities. 7.77 7.90 

14.2 Record effectiveness measures such as time allocated 7.57 7.80 
to training and quality-of-life measures. 

Model transition projects should disseminate 7.13 7.29 

information about their projects by producing a 
product at least at the level of an article for the 
popular press. 

15.1 Employ a project director with a commitment to 7.32 7,33 

disseminate information at least at this level. 
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Outc< 
Number 



Activity 
Nuaber 



Description 



Mean 
Round 1 



Mean 
Round 2 



15.2 Require all workers to keep accurate records cf 

all service and other activities conducted by the 
project. 



7.31 



7.54 



Community Level 
16 



17 



18 



19 



ERLC 



Transition-to-employment projects should develop and 
document a formal interface between education and 
community services (e.g., between schools and state 
vocational rehabilitation agencies). 

16.1 Conduct workshops to train personnel. 

16.2 Communicate needs of project consumers to community 
agency personnel. 

16.3 Document meetings between education professionals and 
professionals/paraprofessionals outside of education. 

Model transition-to-employment projects should accept 
responsibility for forming a state, local, or regional 
taskforce to achieve coordinated leadership and 
direction of a model project. 

17.1 Negotiate directly with administrators from community 
agencies. 

17.2 Conduct public relations programs for targeted audiences, 
such as employers, careproviders , and agencies. 

Model transition-to-employment projects should develop 
and document a cooperative service delivery model when 
more than one agency is providing consumer services. 

18.1 Articulate the roles of all associated agencies. 

18.2 Employ personnel whose role is to coordinate project 
activities. 

18.3 Document services provided by cooperating agencies. 

Model transition-to-employment projects should 
establish and document employment referral services. 

19-1 Operate a database for employer and consumer matching. 

19.2 Conduct and publish regular analyses of job-market 

trends . 



7.72 



7.23 
7.90 

7.13 

6.41 



6.59 
6.97 
7.30 

7.37 
7.00 

7.01 
6.19 

5.81 
4.91 



7.95 



7.40 
8.06 

7.14 

5.95 



6.62 
6.63 
7.54 

7.59 
7.13 

7.05 
6.17 

5.49 
4.63 
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Outcoae Activity Mean Moan 

Hiaiber Muaber Description Round 1 Round 2 



20 Model trans it ion- to -employment projects should produce 6.64 6.44 
expanded rehabilitation services. 

20.1 Document student (client) experiences with rehabilitation 6.65 6.62 
(education) agencies. 

20.2 Develop cooperative activities with rehabilitation 7.47 7.60 
(education) agencies. 

21 Model transition-to-employment projects should 7.90 8.13 
demonstrate improved access to community-based services 

for persons with disabilities. 

21.1 Document the number of students in the project server 7.72 7.82 
by community agencies. 

21.2 Conduct outreach activities such as seminars and 7.52 7.56 
workshops for community agency personnel. 

21.3 Conduct technical assistance services to center-based 7.30 7.31 
agency personnel to foster conversion to community- 
based services. 

22 Model transition-to-employment proejcts should 8.43 8.67 
demonstrate improved work opportunities for youths 

with disabilities. 

22.1 Evaluate and document effectiveness of job placement 8.17 8.56 
and maintenance activities. 

22.2 Research job trends and business requirements. 6.67 6.45 



22.3 Work cooperatively with community agencies to conduct 

longitudinal studies. 



7.39 7.41 



Note . From "Employment Outcomes and Activities for Youths in Transition " by F. R. Rusch, J. 
F. Enchelmaier, and P. D. Kohler, in press, Career Development for Exception al Individuals, 
Copyright by the Division on Career Development and Transition of the Council for 
Exceptional Children. Reprinted by permission. 
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Table 3 



Indicators of Selected Outcomes and Activities Across Four Conceptual Levels 



NUMBER 



DESCRIPTION 




Indicators 

• Hours worked 

• Hourly wages 



• Number of students placed 

• Student demographics 

• Job types 

• Length of employment (days, weeks, months) 

• List of benefits (i.e., medical, vacation, sick time, profit sharing) 

• Number of jobs held prior to graduation and summary of evaluations of each 
placement 

• Annual salary 

• Number of full-time and part-time positions 

• Job patterns (e.g., never changed, voluntary change, laid off, quit, etc.) 

• Student satisfaction measures 

• Termination, reason (elaborate) 

• Record of how job was initially obtained and by whom 

• Number of employees at each business 

• Number and ratio of employees without disabilities on site when student present 

• Record of job match to student-stated work interests 

• Record of job advancement following initial placement -- increased job 
responsibilities or increased wages (e.g., in hrs worked, raises, promotions, job task 
responsibilities; increased level of indispensability to employer) 

• Documentation of single-subject research study 

• Reduction in public assistance (e.g., SSI, workers comp, public aid, etc.) 



Indicators 

• Potential Employer Contact Log (# of entries) 

• Number of potential employers 

• Number of student interviews 

• Number of placements 

• Types of placements 

• Number of potential employees 

• Identification of natural supports in each placement 

• Documentation of a specific "marketing" plan for each student 

• Student/family job preferences 

• Number employers contacted and method 

• Number student interviews, alone or assisted 

• Number of successful and unsuccessful placements and whether placement was in 
the top-priority list of student or family 

• Trainee waiting lists for placement 

• Number of placements per student 



Activity la 



Provide job placement services. 
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NUMBER 



DESCRIPTION 



Activity lb 



Work with aduit service agencies to ensure job placement 



Indicators 

• Signature of adult service representatives on each IEP 

• Record or letter of interagency agreement 

• Referral documentation 
Number of meetings and frequency 
Record of joint projects (i.e., Job Fair) 
Record of business involvement with adult service agencies 
Identification of roles adult service agencies play in job placement 
Log of case management activities by type and frequency 
Record of transition plan updates with adult service agencies 
Written statement of percent time commitment to student and family that will 
occur at age 21 

Number of meetings of adult agencies with school personnel and/or student/ 
family 

Written progress reports on placement efforts 
List of transition team members 



Activity lc Provide job exploration and job training opportunities as part of the school 

curriculum to prepare students for competitive employment. 

Indicators 

• Number of days training in community (unpaid) per week 

• Number of days paid work per week 

• Hours worked during school day 

• Record of training and employment sites 

• Record of peer job coaching 

• Record of gifted and talented student support 

• Record of all school personnel involved in curriculum 

• Number of hours 

• List of job exploration sites, general and specific purposes for each site, 
competencies gained per student during exploration, number of hours in job 
exploration, and type of site 

• Performance data per student 

• Data on level of supervision 

• List of products produced and quantity (e.g., student resumes, training plans, or 
work profiles) 

• Number of training sites per student 

• Documentation of individualized training programs for students that reflect 
systematic instruction and strategics 

• Baseline and probe data pertaining to training 

• Graphs of student performance 

• Documentation of types of instruction or training provided 
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NUMBER DESCRIPTION 



Activity Id Provide job support services. 

Indicators 

• Number of training hours provided by teacher 

• Number of hours teacher present at employment site 

• Record of off-site support provided by teacher 

• Number of observation hours provided by teacher 

• Record of support provided by others (i.e., job coaches, voc. rehab, counselors, job 
developers) 

• Number of hours by school personnel other than teachers 

• Record of support provided by parents 

• Record of support provided by employer or supervisor 

• Contacts with family 

• Hours and type of school-based job-related instruction 

• Record of data-based fading against targeted performance criteria 

• Number of employee evaluations done by employer per month 

• Record of support provided by co-workers 

• Record of unusual incidents and training interventions 

• Record of training strategies and procedures 

• Record of data collection documenting skill acquisition 



Activity le Provide technical assistance to adult service agencies to provide job placement and 

job support services. 

Indicators 

• In-service workshop hours directed toward teaching job-placement and support 
techniques 

• Hours of direct training provided to job coaches on the job 

• One-to-one contact (contact logs) 

• Evidence of curriculum used 

• Type of training and technical assistance activities provided 

• Record of funds spent to train adult service providers 

• Workshop or inservice evaluation data 

• Needs assessment data 

• Number of people trained 

• List of training materials developed 
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NUMBER DESCRIPTION 



PROGRAM LEVEL 

Outcome 6 Model transition projects should utilize individualized transition planning for 

students with disabilities. 

Indicators 

• Number of plans developed 

• Number of plans completed 

• Written documentation of transition services as part of IEP 

• Record of number of transition planning meetings 

• Signatures of adult service providers and community agency personnel on IEP 

• Compilation of types of transition outcomes for students on annual basis 

• Results of parent or family survey of planning process 

• Evidence from student files illustrating program modifications to meet student's 
individualized transition needs 

• List of agency representatives participating in planning 

• Number of transition objectives in IEP 

• Documentation of assessment information utilized in plan development 

• Documentation of parent involvement in planning 

• Documentation of student involvement in planning 

• Documentation that service or curricular activities were provided as stated in the 
student's plan 

• Evidence of curriculum that facilitates student involvement in planning 

Activity 6a Develop strong cooperative linkages with vocational rehabilitation services to 

develop the IEPs. 

Indicators 

• Number of agreements developed 

• Number of consultations 

• Documented participation of personnel 

• Signature of vocational rehabilitation personnel on IEP 

• Record of contacts between vocational rehabilitation and student 

• Report of number of students receiving services from vocational rehabilitation 

• Letters of agreement 

• Documentation of joint use of information (e.g., school records, assessment 
information, medical data) 

• Evidence of referral system or process 

• Documentation of services provided to students 
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NUMBER 



ERIC 



DESCRIPTION 



Activity 6b Develop strong cooperative linkages with vocational education services to develop 

the IEPs. 

Indicators 

• Record of agreement(s) developed 

• Number of joint activities, meetings, or consultations 

• Record of actual participation in 1EP development 

• Signature of vocational education personnel on IEPs 

• Report of number of goals or objectives contained in IEP carried out by or in 
conjunction with vocational education 

• Record of vocational education contact with families 

• Vocational education services identified on IEP 

• Record of attendance at meetings 

• Record of interagency agreements 

• Record of student enrollment in vocational curricula 

• Record of vocational education services provided 

• Evidence of collaborative consultation between voc ed and special ed 
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NUMBER 



DESCRIPTION 



ORGANIZATION LEVEL 



Outcome 15 



Activity 15a 



Activity 15b 



Activity 15c 



Transition-to-employment projects should develop and document a cooperative 
service delivery model when more than one agency is providing consumer services. 

Indicators 

• Number of agencies providing services 

• Organizational structure for service provision 

• Interagency agreement(s) 

• Record of referral among agencies 

Articulate the roles of all associated agencies. 
Indicators 

• Evidence of a process for developing of collaborative agreements 

• Documentation of collaborative agreements 

• Documentation of contractual arrangements 

• Evidence of a process reviewing roles on a regular basis 

• Number of formal operating agreements 

• Number of informal operating agreements 

• Evidence of networking effectiveness 

• Evidence of a designated "coordinating" agency to oversee local agencies 

• Number or existence of state laws or regulations reducing barriers to agency 
collaboration or articulating collaborative roles 

• Documentation of services provided by and efforts of each agency 

Employ personnel whose role is to coordinate project activities. 
Indicators 

• Job description of project coordinator or manager 

• Evidence of a "coordinating 1 * agency to oversee local agencies 

• Documentation of state and local funds earmarked to support coordinating agency 
in this task 

• Evidence of agency and project funding of personnel 

Document services provided by cooperating agencies. 
Indicators 

• Record of employment services provided by type and by student 

• Record of community living facilities and/or services provided 

• Record of transportation arrangements and services 

• Assignment of coordinating personnel, agency, or local planning councils to collect 
specific data on services offered, clients, costs, etc. 

• Analysis of data collected for future decision making 

• Case history of clients 

• Number and type of activities completed by caseworkers 

• Evidence of a process for evaluating accountability of cooperating agencies 
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NUMBER 



DESCRIPTION 



COMMUNITY LEVEL 



Outcome 17 



Activity 17a 



Activity 17b 



Model transition-to-employment projects should demonstrate improved work 
opportunities for youths with disabilities. 

Indicators 

• Record of types of jobs that comprise placements 

• Statistics pertaining to wages, benefits, and hours 

• Percent of graduating class employed by level of employment (i.e., full-time, part- 
time) 

• Percent employed at or above minimum wage 

• Percent who move to improved work situations (e.g., promotions, job changes for 
increased salary, benefits, working hours, etc.) 

• Percent who lose jobs and/or move to "poorer" jobs 

• Number of employers associated with project or program 

• Number of students employed first year of project compared to subsequent years 

Evaluate and document effectiveness of job placement and maintenance activities. 
Indicators 

• Length of time on job 

• Employee satisfaction data concerning job placement, pre-/post- project 

• Employer satisfaction data concerning job placement, pre- /post- project 

• Documented opportunities for advancement 

• 1-, 3-, 5-year follow-ups on youths: 

1. Employed in jobs for which training was provided; in jobs for which training 
was not provided 

2. Employed but changed job (up and down) in job trained; not in jobs trained 

3. Unemployed; never employed; previously employed 

• Data on youths employed and wages, benefits, length of employment, pre-/post- 
project 

• Data on family satisfaction with job placement, pre-/post- project 

• Data on employer willingness to hire, pre-/post- project 

Research job trends and business requirements* 
Indicators 

• Record of project or employer advisory committee, members, meetings 

• Labor-market surveys: 

Stable employment opportunities 

Potential increased employment opportunities 

Decreasing employment opportunities 

Dead-end employment 

Career ladder employment 

Job requirements 
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NUMBER 



DESCRIPTION 



Activity 17c 



Work cooperatively with community agencies to conduct longitudinal studies. 



Indicators 

• Assess student outcomes: employment, community living, recreation, etc. 

• Assess quality of life via self rating or reliable informant 

• Follow-up data on individuals who change agencies 

• Evaluation data pertaining to working relationship between agencies and project 

• Record of agreements detailing longitudinal studies to be conducted and roles of 
participants 

• Documentation of funding source 

• Evidence of research design for study 



Note . From "School to Work Transition: Identification of Employment-Related Outcome and 



Activity Indicators" by P. D. Kohler and F. R. Rusch, 1993, Transition Research Institute, 



University of Illinois (submitted for publication). Reprinted by permission. 
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Organization 



Community 




Figure 1. Systems-level conceptual framework for evaluating program activities and outcomes. 



Note . From "School to Work Transition: Identification of Employment-Related Outcome and 
Activity Indicators" by P. D. Kohler and F. R. Rusch, 1993, Transition Research Institute, 
University of Illinois (submitted for publication). Reprinted by permission. 
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Outcome 

General statement of results; 
the product of action 




Activity 

Specific statement; 
action associated with 
producing outcome(s) 



-/indie 



Indicators ) that activity 
has taken place 




Activity 

Specific statement; 
action associated with 
producing outcome(s) 



— /indicator(s) that activity} 
v^iias taken place^X 



Activity 

Specific statement; 
action associated with 
producing outcome(s) 




Indicator(s) that activity 
has taken place 



Fi gure 2 . Analytical model illustrating perceived organizational relationship between an 
outcome, activities, and indicators. 

Note. From "School to Work Transition: Identification of Employment-Related Outcome and 
Activity Indicators" by P. D. Kohler and F. R. Rusch, 1993, Transition Research Institute, 
University of Illinois (submitted for publication). Reprinted by permission. 
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